Skip to content

Refactor GitlabDataSource to work with browser extension - #1524

Merged
keshav-space merged 5 commits into
aboutcode-org:mainfrom
michaelehab:vulntotal-change-gitlab-datasource
Aug 19, 2024
Merged

Refactor GitlabDataSource to work with browser extension#1524
keshav-space merged 5 commits into
aboutcode-org:mainfrom
michaelehab:vulntotal-change-gitlab-datasource

Conversation

@michaelehab

Copy link
Copy Markdown
Collaborator

Fix #1523

This PR streamlines the GitLab datasource by eliminating the need for retrieving the repo subtree as a tar.gz file. Instead, it directly fetches YAML files from the repository. This change enhances compatibility, especially in Chrome extensions, and simplifies the codebase.

Changes Made:

  • Removed tar.gz retrieval logic.
  • Implemented direct YAML file fetching.

Testing:

  • Verified functionality in various scenarios, including cli and Chrome extension usage.

@michaelehab
michaelehab requested a review from keshav-space July 24, 2024 13:27
@michaelehab
michaelehab force-pushed the vulntotal-change-gitlab-datasource branch 2 times, most recently from a3583f4 to 569e2f2 Compare July 24, 2024 15:42
@keshav-space keshav-space added the VulnTotal Tool for cross-validating vulnerability label Jul 29, 2024

@keshav-space keshav-space left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @michaelehab!
See some nits below and add DCO to your commits.

Comment thread vulntotal/datasources/gitlab.py

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the test code, I read the contents of temp_vulntotal_gitlab_datasource folder, and this file is not included in the parsed_advisory-expected so the test was failing when we had it

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread vulntotal/datasources/gitlab.py Outdated
Comment thread vulntotal/datasources/gitlab.py
@michaelehab
michaelehab force-pushed the vulntotal-change-gitlab-datasource branch from 3761d09 to f5a1163 Compare August 3, 2024 18:20

@keshav-space keshav-space left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelehab You need to properly rebase or merge with the main branch without modifying the original commits. Let me know if you need help with this.

-Update Gitlab Datasource tests

-Reorder test_parse_interesting_advisories test files

-Format changed files with black

-Merge fetch yml logic and update method doc

Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>
@michaelehab
michaelehab force-pushed the vulntotal-change-gitlab-datasource branch from e2298be to 8bc0445 Compare August 8, 2024 17:39
@michaelehab

Copy link
Copy Markdown
Collaborator Author

@keshav-space I squashed the changes and properly merged them with the main branch. Thank you!

Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>
Comment thread vulntotal/tests/test_gitlab.py
Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>

@keshav-space keshav-space left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @michaelehab, Looking good!

@keshav-space
keshav-space merged commit d62f377 into aboutcode-org:main Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

VulnTotal Tool for cross-validating vulnerability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simplify GitLab Datasource by Directly Fetching YAML Files

2 participants